CLOI-NET: Class segmentation of industrial facilities’ point cloud datasets

Shape segmentation from point cloud data is a core step of the digital twinning process for industrial facilities. However, it is also a very labor intensive step, which counteracts the perceived value of the resulting model. The state-of-the-art method for automating cylinder detection can detect cylinders with 62% precision and 70% recall, while other shapes must then be segmented manually and shape segmentation is not achieved. This performance is promising, but it is far from drastically eliminating the manual labor cost. We argue that the use of class segmentation deep learning algorithms has the theoretical potential to perform better in terms of per point accuracy and less manual segmentation time needed. However, such algorithms could not be used so far due to the lack of a pre-trained dataset of laser scanned industrial shapes as well as the lack of appropriate geometric features in order to learn these shapes. In this paper, we tackle both problems in three steps. First, we parse the industrial point cloud through a novel class segmentation solution (CLOI-NET) that consists of an optimized PointNET++ based deep learning network and post-processing algorithms that enforce stronger contextual relationships per point. We then allow the user to choose the optimal manual annotation of a test facility by means of active learning to further improve the results. We achieve the first step by clustering points in meaningful spatial 3D windows based on their location. Then, we apply a class segmentation deep network, and output a probability distribution of all label categories per point and improve the predicted labels by enforcing post-processing rules. We finally optimize the results by finding the optimal amount of data to be used for training experiments. We validate our method on the largest richly annotated dataset of the most important to model industrial shapes (CLOI) and yield 82% average accuracy per point, 95.6% average AUC among all classes and estimated 70% labor hour savings in class segmentation. This proves that it is the first to automatically segment industrial point cloud shapes with no prior knowledge at commercially viable performance and is the foundation for efficient industrial shape modeling in cluttered point clouds.     

Food recognition using neural network classifier and multiple hypotheses image segmentation

ABSTRACT

This paper proposes the multiple-hypotheses image segmentation and feed-forward neural network classifier for food recognition to improve the performance. Initially, the food or meal image is given as input. Then, the segmentation is applied to identify the regions, where a particular food item is located using salient region detection, multi-scale segmentation, and fast rejection. Then, the features of every food item are extracted by the global feature and local feature extraction. After the features are obtained, the classification is performed for each segmented region using a feed-forward neural network model. Finally, the calorie value is computed with the aid of (i) food volume and (ii) calorie and nutrition measure based on mass value. The experimental results and performance evaluation are validated. The outcome of the proposed method attains 0.947 for Macro Average Accuracy (MAA) and 0.959 for Standard Accuracy (SA), which provides better classification performance.     

基于深度语义分割的多源遥感图像海面溢油监测

针对遥感图像海面溢油区域通常受到斑噪声以及强度不均等因素的影响,从而导致溢油区域监测效果较差的问题,本文引入了深度语义分割的方法,将深度卷积神经网络与全连接条件随机场相结合,形成端对端连接。以Resnet结构为基础,首先通过深度卷积神经网络对多源遥感图像粗分割并作为输入,然后经过改进的全连接条件随机场,利用高斯成对势和平均场近似定理,建立条件随机场形成递归神经网络作为输出。通过多源遥感图像对海面溢油区域进行监测,并利用可见光图像估计溢油区域面积。实验在所建立的多源遥感图像数据集上与其它先进模型进行对比,结果表明本文方法提高了溢油区域的分割精度以及精细细节程度,平均交并比为82.1%,监测效果具有明显地改善。     

Microscopic melanoma detection and classification: A framework of pixel‐based fusion and multilevel features reduction

The numbers of diagnosed patients by melanoma are drastic and contribute more deaths annually among young peoples. An approximately 192,310 new cases of skin cancer are diagnosed in 2019, which shows the importance of automated systems for the diagnosis process. Accordingly, this article presents an automated method for skin lesions detection and recognition using pixel‐based seed segmented images fusion and multilevel features reduction. The proposed method involves four key steps: (a) mean‐based function is implemented and fed input to top‐hat and bottom‐hat filters which later fused for contrast stretching, (b) seed region growing and graph‐cut method‐based lesion segmentation and fused both segmented lesions through pixel‐based fusion, (c) multilevel features such as histogram oriented gradient (HOG), speeded up robust features (SURF), and color are extracted and simple concatenation is performed, and (d) finally variance precise entropy‐based features reduction and classification through SVM via cubic kernel function. Two different experiments are performed for the evaluation of this method. The segmentation performance is evaluated on PH2, ISBI2016, and ISIC2017 with an accuracy of 95.86, 94.79, and 94.92%, respectively. The classification performance is evaluated on PH2 and ISBI2016 dataset with an accuracy of 98.20 and 95.42%, respectively. The results of the proposed automated systems are outstanding as compared to the current techniques reported in state of art, which demonstrate the validity of the proposed method.     

Automatic brain tissue segmentation in MR images using hybrid atlas forest based on confidence-weighted probability matrix

The segmentation of specific tissues in an MR brain image for quantitative analysis can assist the disease diagnosis and medical research. Therefore, a robust and accurate method for automatic segmentation is necessary. Atlas-based-method is a common and effective method of automatic segmentation where an atlas refers to a pair of image consist of an intensity image and its corresponding label image. Apart from the general multi-atlas-based methods, which propagate labels through the single atlas then fuse them, we proposed a hybrid atlas forest based on confidence-weighted probability matrix to consider the atlases set as a whole and treat each voxel differently. In the framework, we first register the atlas to the image space of target and calculate the confidence of voxels in the registered atlas. Then, a confidence-weighted probability matrix is generated and it augments to the intensity image of the atlas or target for providing spatial information of the target tissue. Third, a hybrid atlas forest is trained to gather the features and correlation information among the atlases in the dataset. Finally, the segmentation of the target tissues is predicted by the trained hybrid atlas forest. The segment performance and the components efficiency of the proposed method are evaluated on the two public datasets. Based on the experiment results and quantitative comparisons, our method can gather spatial information and correlation among the atlases to obtain an accurate segmentation.     

Retinal Vessel Extraction Framework Using Modified Adaboost Extreme Learning Machine

An explicit extraction of the retinal vessel is a standout amongst the most significant errands in the field of medical imaging to analyze both the ophthalmological infections, for example, Glaucoma, Diabetic Retinopathy (DR), Retinopathy of Prematurity (ROP), Age-Related Macular Degeneration (AMD) as well as non retinal sickness such as stroke, hypertension and cardiovascular diseases. The state of the retinal vasculature is a significant indicative element in the field of ophthalmology. Retinal vessel extraction in fundus imaging is a difficult task because of varying size vessels, moderately low distinction, and presence of pathologies such as hemorrhages, microaneurysms etc. Manual vessel extraction is a challenging task due to the complicated nature of the retinal vessel structure, which also needs strong skill set and training. In this paper, a supervised technique for blood vessel extraction in retinal images using Modified Adaboost Extreme Learning Machine (MAD-ELM) is proposed. Firstly, the fundus image preprocessing is done for contrast enhancement and in-homogeneity correction. Then, a set of core features is extracted, and the best features are selected using “minimal Redundancy-maximum Relevance (mRmR).” Later, using MAD-ELM method vessels and non vessels are classified. DRIVE and DR-HAGIS datasets are used for the evaluation of the proposed method. The algorithm’s performance is assessed based on accuracy, sensitivity and specificity. The proposed technique attains accuracy of 0.9619 on the DRIVE database and 0.9519 on DR-HAGIS database, which contains pathological images. Our results show that, in addition to healthy retinal images, the proposed method performs well in extracting blood vessels from pathological images and is therefore comparable with state of the art methods.     

一种结合区域检测和语义分割的SLAM技术

提出一种结合区域检测和语义分割的即时定位和建图（SLAM）技术，通过引入高精度图像描述子SIFT来实现前端视觉里程计（VO）过程中帧间像素匹配的精度。为了降低引入操作带来的计算复杂度，设计一个实时区域检测算法，在相邻帧间检测大致相似的ROI(Region of Interest)关键区域，使得SIFT描述子的提取和匹配只在ROI区域内完成，其余区域仍旧采用精度略低、效率更高的ORB算子。同时，为了提高后端BA(Bundle Adjustment)的精度，减少累积误差，结合语义图，在原有的基本投影误差函数上添加一个语义误差。该语义图采用实时语义分割算法完成，同时只针对ROI区域进行分割。通过与原SLAM方案对比实验，表明本文提出的方法，在提高一定精度的同时，仍能满足SLAM实时定位和建图的要求。最后，在电力作业场景下验证了该方案的效果。     

磁环表面缺陷图像分割算法的研究及实时实现

为实现小型磁环表面细微缺陷图像无监督分割,并提高分割精度与计算效率,本文提出了一种基于改进2D Gabor滤波器组的自适应阈值分割方法。首先,利用多尺度、多方向的Gabor滤波器组对缺陷图像进行滤波降噪处理,抑制目标区域与背景区域内部的噪声污染,同时增强区域间的差异性；然后,通过对处理后图像的灰度统计特性分析,根据缺陷图像的灰度均值及方差构造了灰度阈值计算公式,实现了小型磁环表面细微缺陷图像的自适应分割。实验结果表明,本文算法可快速、准确地分割缺陷并抑制噪声干扰,在分割精度、计算效率等方面也优于传统的选择迭代法、OTSU、最大熵等方法,并能够在先进的SEED-DVS8168平台上实时实现,验证了此算法的可行性与实时性。     

中低温煤焦油基炭微球的制备

通过馏分切割、温和加氢相结合对中低温煤焦油进行精制处理，精制后的原料采用分级热聚制备中间相炭微球。考察了精制处理条件对原料性质、中间相炭微球宏观外貌及微晶结构的影响。采用FTIR、GC-MS、族组成、元素分析对原料进行表征，采用SEM、XRD对中间相炭微球进行表征。结果表明：中低温煤焦油中300～430℃馏分油是制备中间相炭微球的较佳馏分。300～430℃馏分油中正庚烷可溶物（HS）质量分数高达84.76%，吡啶不溶物（PI）质量分数低至0.23%，杂原子含量低，芳烃化合物的环数为2～4环。300～430℃馏分油在T_H=350℃、p=8MPa、t=1.5h、剂油比1∶40（质量比）的条件下温和加氢得到的精制原料，经420℃热聚6h得制备的中间相炭微球宏观外貌、微晶结构较好。中低温煤焦油基炭微球的粒径范围为5～15μm，小球表面光滑，微观结构为地球仪型，经1450℃高温煅烧后，石墨化度达到12.33%。